1. Field of the Invention
The present invention relates to a technique in which images of a subject is picked up from a plurality of directions to form a three-dimensional image model of the subject.
2. Description of the Background Art
Images of a three-dimensional subject are picked up from a plurality of directions, and a plurality of pieces of the resulting image data are composed so as to form a three-dimensional image model of the corresponding subject. In other words, when data such as camera external parameters (camera position, orientation, etc.) and inner parameters (focal length, etc.) are obtained for each of the respective images obtained by image-picking up the subject from a plurality of directions, a three-dimensional model of the subject is re-composed from silhouette images of the subject by using a Shape-From-Silhouette method. The detailed description of the Shape-From-Silhouette method is given in W. Niem, “Robust and Fast Modelling of 3D Natural Objects from Multiple Views” SPIE Proceedings Image and Video Proceeding II vol. 2182, 1994, pp. 388-397. Hereinafter, the camera external parameters and inner parameters are generally referred to as “(camera) calibration parameters”, and in the case when, among these calibration parameters, the inner parameters have been known and the camera calibration processes using the inner parameters have been completed, if the external parameters of the camera are found, it is possible to compose a three-dimensional image model of the subject.
Here, one of the methods for image-picking up a subject from a plurality of directions in this manner is a fixed position method in which a plurality of cameras are placed in different positions in a fixed manner and images of the subject are taken from these positions. However, in this fixed position method, since a plurality of cameras need to be fixedly placed in a dispersed manner inside an image-pickup studio, etc., it is necessary to provide a complicated, large image-pickup facility.
Therefore, a moving image-pickup method have been proposed, in which the user successively picks up images of a subject from a plurality of directions while he or she moves around the subject with a portable camera in the hands so that all the circumferential images of the subject are obtained.
However, in this moving image-pickup method, in order to determine the external parameters of the camera, it is necessary to specify the camera position and orientation in each of the image pickup operations.
With respect to the method of measuring external parameters of a camera for such purposes, conventional methods, such as a magnetic method, an ultrasonic method and an optical method, have been proposed. Among these, in the magnetic method, the terrestrial magnetism, etc., at the camera position are detected so that the positions and orientations of the respective cameras are specified, and in the ultrasonic method, ultrasonic waves from a predetermined ultrasonic wave are detected so that these factors are specified. In the optical method, there are, for example, methods in which a stereophonic camera is used and a calibration chart which is greater than the field of view is placed.
Among these methods, in the magnetic method, it becomes difficult to measure with high precision when the subject is made of metal, and the ultrasonic method results in high costs of the apparatus.
Moreover, techniques for finding the relative position and relative orientation of a person and an object within a space has been developed in the field of virtual reality, etc. For example, an algorithm for finding these factors has been disclosed in “L. Quan, Z. Lan, “Linear N-Point Camera Pose Determination”, IEEE Trans. PAMI 21(8) 1999” and “Takahashi, Ishii, Makino, Nakashizu, “A Measuring Method of Marker Position/Orientation for VR Interface by Monocular Image Processing”, Electronic Information Report Journal AJ79 1996”. In accordance with these techniques, when a reference object is image-picked up, the picked-up image is used to find the relative position and relative orientation of the image-pickup position. Hereinafter, the algorithm, disclosed in these documents, is referred to as “multipoint analyzing algorithm”.
Here, with respect to a conventional optical method, Japanese Laid-Open Patent Application No. 2000-270343 has disclosed a method in which a single plane chart on which an irregular matrix pattern is drawn is placed at a predetermined position, and by observing this with a camera, the relationship of the position and orientation between the camera and the plane chart are specified by using the “multipoint analyzing algorithm”. In this method, since the relative relationship of the position and orientation of the camera with respect to a coordinate system fixed to the plane chart is obtained; therefore, when the relationship of the position and orientation between the plane chart and the subject is fixed so that, each time the subject is image-pick up from each of a plurality of directions, the pattern on the plane chart is observed by the camera so as to specify the position and orientation of the camera at that point on the absolute coordinate system.
However, the plane chart has only a narrow angle range from which it can be observed, and it cannot be observed from directions exceeding 90 degrees from the normal direction of the plane chart; therefore, the camera movable range is extremely limited. Moreover, even in the case when a camera is located within a range from which the plane chart can be observed, if the camera direction is greatly deviated from the normal direction of the plane chart, there is a reduction in the observation precision of the pattern, and the resulting problem is that the camera external parameter determining precision becomes poor.
In particular, when a subject is an object that is considerably larger than the plane chart, the conventional optical method fails to obtain accurate external parameters, resulting in a failure in composing a three-dimensional image model of the subject.
Moreover, in the technique of Japanese Laid-Open Patent Application No. 2000-270343, it is uncertain whether or not the obtained image has sufficient precision to be subjected to calculation processes at the time of the image-pickup process, and it is not possible to determine whether or not the image-pickup operation can be continued under the current state as it is; therefore, wasteful image-pickup operations tend to be carried out.